-
-
Notifications
You must be signed in to change notification settings - Fork 541
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: Update linters and essential configurations #743
base: master
Are you sure you want to change the base?
Conversation
for more information, see https://pre-commit.ci
@webknjaz is there a way to make As it generates changes for all these files, when none of them were actually touched in this PR |
This is actually good — new linters should include changes to the linted things. This makes such changes atomic. |
types_or: [python, pyi] | ||
- id: ruff-format | ||
types_or: [python, pyi] | ||
args: [--config, format.quote-style = 'single', --config, line-length = 100] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There should be a good reason(s) to deviate from linter's default settings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not because of the defaults (for which sometimes there are good reasons to change) but because other tools wouldn't pick this up, it should be avoided (with a few well-defined exceptions like the one I've got for MyPy).
For this reason, every low-level tool must be configured via its own autoloaded config file name. Otherwise, everybody would have to duplicate this config in their editors, IDEs, other tools and sometimes CI, and would have toe know upfront that this is necessary. Being surprised by unexpected behaviors when they don't. I have some notes on this in #742.
I wouldn't say that setting such a long line setting has a merit, though. I'm a firm believer that in the name of readability/inclusivity/maintainability, code should be readable in columns. Just like any text, typography can govern column width for code too. This is directly connected to readability (which is why magazines and newspapers don't print lines from one side of the page to the others, and some books / catalogs use columns). Typographically optimal columns are between 50 and 75 chars (opinions vary, but mostly close to this range): https://baymard.com/blog/line-length-readability. We have a long-standing convention of 79 that exceeds it a bit, but isn't as critical. It allows for columns in editors for working on multiple files while still using enlarged fonts. It also allows for side-by-side diffs. The lines fit on the screen in these settings as well as allow for column-based top-down reading (which is another thing humans do with text naturally, by habit).
args: | ||
- -i | ||
- --max-line-length=100 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Out of interest: why do you sometimes use args: […]
and sometimes split args
arrays to multiline? Such inconsistency is a bit confusing as it adds impression that this is done for a reason.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sometimes it depends on how hook configured - IE
[--config, format.quote-style = 'single', --config, line-length = 100]
will be actually more readble as
[
--config, format.quote-style = 'single',
--config, line-length = 100
]
not
- --config
- format.quote-style = 'single'
- --config
- line-length = 100
But I don't remember why I can't use multiline array with []
, maybe I just not set it in first place when set it a long time ago.
Second reason - our God of code reusage across projects - copy-paste :D
Nobody never reevaluated it, so it is as it is till now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is rather subjective. I'd argue that []
is the least readable way.
Actually, for --long-args
, it's best to use =
not a whitespace. This is useful not only in this config but in Python code too. Since you're integrating autoformatters, they will produce such weird constructs with args disconnected from their values unless you use equals that bundles them into the same string.
hooks: | ||
- id: mypy | ||
additional_dependencies: | ||
- types-PyYAML |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why four rather than two spaces indentation at this line? 😕
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yamllint all the things!
I'd add https://github.com/ansible/awx-plugins/blob/53ebc59/.yamllint with quoted-strings: {required: only-when-needed}
on top (https://yamllint.readthedocs.io/en/stable/rules.html#module-yamllint.rules.quoted_strings).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
|
||
|
||
########## | ||
# PYTHON # |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With pre-commit, the primary ordering should be by whether something is faster or slower. Putting instant checks at the beginning improves responsiveness, with the slow ones stuffed at the end. The second factor is formatters vs. linters. When a bunch of formatters change some type of files, they should be executed before the linters that check the same files.
The ecosystem is something that could be used to bundle similar checks within those groups. But it's definitely not something that would be top level.
hooks: | ||
- id: pylint | ||
args: | ||
- --disable=import-error # E0401. Locally you could not have all imports. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Avoid putting linter settings into CLI args at all costs. This is incompatible with literally everything that would run said linters. Always keep them in tool-specific configs.
- --disable=fixme # W0511. 'TODO' notations. | ||
- --disable=logging-fstring-interpolation # Conflict with "use a single formatting" WPS323 | ||
- --disable=ungrouped-imports # ignore `if TYPE_CHECKING` case. Other do reorder-python-imports | ||
- --disable=R0801 # Similar lines in 2 files. Currently I don't think that it possible to DRY hooks unique files boilerplate |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't use numeric rule codes in pylint. It has human-readable names that make much more sense than random digit sequences.
Also, instead of disabling the rules, many pylint and flake8 rules can remain enabled because they allow tweaking their settings. I know for a fact that this duplication rule allows increasing the number of lines to take into account.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
I can only pray that pylint
maintainers will add human-readable names in their error msgs, and not require users to google if they exist and how they named if so
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have this configured by default 🤷♂️ You should've just used my config.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately, Ruff does not allow using those names in ignores, while the rules may have human-readable names. There's a feature request about it.
args: | ||
- --disable=import-error # E0401. Locally you could not have all imports. | ||
- --disable=fixme # W0511. 'TODO' notations. | ||
- --disable=logging-fstring-interpolation # Conflict with "use a single formatting" WPS323 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Drop this. Logging actually shouldn't be using f-strings.
hooks: | ||
- id: mypy | ||
additional_dependencies: | ||
- types-PyYAML |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yamllint all the things!
I'd add https://github.com/ansible/awx-plugins/blob/53ebc59/.yamllint with quoted-strings: {required: only-when-needed}
on top (https://yamllint.readthedocs.io/en/stable/rules.html#module-yamllint.rules.quoted_strings).
--ignore-missing-imports, | ||
--disallow-untyped-calls, | ||
--warn-redundant-casts, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nothing should be disabled globally. Suppressions must be granular, which can be implemented in the config and in-module.
- --max-imports=15 # Default to 12 | ||
# https://wemake-python-stylegui.de/en/latest/pages/usage/violations/index.html | ||
- --extend-ignore= | ||
E501 <!-- line too long (> 79 characters). Use 100 --> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Configurable rules shouldn't be ignored really.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- It covered by other linters, so deduplicate similar errors
- Idk why it not provide link to docs which describes possibilities how to deal with violation as many hooks do 🤔 It's not easy to google them. This one https://www.flake8rules.com/rules/E501.html doesn't include any suggestions about configuration -_-
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WPS can show shortlinks to the rules, you just didn't enable it 🤷♂️
This is the setting: https://flake8.pycqa.org/en/latest/user/options.html#cmdoption-flake8-max-line-length.
But anyway, it doesn't make sense to have such things as CLI args. I can send in a good config structure where it's more apparent what should go where.
'Programming Language :: Python :: 2', | ||
'Programming Language :: Python :: 2.7', | ||
'Programming Language :: Python :: 3', | ||
'Programming Language :: Python :: 3.6', | ||
'Programming Language :: Python :: 3.7', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's best to list envs tested in CI here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's usually best to couple formatting changes with adding a formatter. Not multiple automatic and manual formatting changes with formatters and linters all smashed together. It's difficult to figure out what's coming from where and why that is in this setting.
Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]> Co-authored-by: George L. Yermulnik <[email protected]>
- --max-line-length=100 | ||
|
||
# Usage: http://pylint.pycqa.org/en/latest/user_guide/message-control.html | ||
- repo: https://github.com/PyCQA/pylint |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I experimented with configuring this linter for a while, but I'm hitting a few bugs within.
You're also integrating Ruff which reimplements most if not all of pylint rules in it.
It doesn't really make sense to have multiple linters report the same violation multiple times within each pre-commit
run.
With that in mind, I recommend abandoning pylint
for now and focusing on making sure all the corresponding rules in Ruff are enabled instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I guess it's a worthwhile suggestion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FWIW, I'm planning to research replacing many things with Ruff in my projects but just didn't get to do a full comparison.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, ruff has rules that should be manually enabled? Didn't know it
From what I saw - pylint in its current configuration provides additional checks which are not covered by ruff
in its current configuration.
At the same time, it totally makes sense to check is ruff
and we-make-styleguide
both fully cover pylint
out of the box or at least able to be configured in such way.
If you do such research - that's would be lovely, but till I want to have all them in place, as better to catch all possible issues now, than deal with them in 6-12months when they detection will be implemented in ruff
etc.
pylint
check is relatively fast, so I want to preserve it for now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pylint
check is relatively fast
😆 this is the slowest checker I know. This is because it goes beyond static analysis and does some dynamic checking as well.
As for ruff, I wanted to research it more. And yes, there's rules that are disabled by default in all the linters I've ever met (flake8, pylint, ruff). Some updates to pylint sometimes move the rules from default to extensions too — this is why I prefer an explicit config to make sure that it doesn't suddenly stop checking something just because of a version bump.
While I do like pylint in general, I tried it locally on this codebase and there's bugs in it that block really adding it in full capacity. Hence, the idea to delay adding it. I may end up debugging the issue for my other projects. But in general, many people stick with flake8 because it's not as slow and easier to extend. We do run it in ansible-test
, though.
My initial strategy for adopting Ruff would be enabling as much as possible there, and only if some rule is not ported, then run flake8/pylint (with plugins) only checking those rules, to reduce the overhead. It's good for DX when the same problem isn't reported by 5 different checkers.
- repo: https://github.com/pre-commit/mirrors-autopep8 | ||
rev: v2.0.4 | ||
hooks: | ||
- id: autopep8 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With Ruff in formatting mode, this is probably unnecessary. I'm yet to compare their output, but the recommendation is going to be the same as with pylint for now.
name: isort | ||
args: [--force-single-line, --profile=black] | ||
|
||
- repo: https://github.com/asottile/add-trailing-comma |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This one might be fine to combine with Ruff. But you'll have to put it first and also check if they don't cancel each other out in the way they're configured.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They don't
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They don't
I wouldn't be so sure. I've seen corner cases with various formatters where they'd do this very occasionally under certain conditions. It's good that we haven't observed such a behavior here. I'm just saying that this should be kept in mind for the future in case it does happen.
If Ruff does not implement a similar capability, it stands to reason that it's okay to keep this formatter. If it does, it might not make sense.
But if you decide to keep this one, make sure to move it along with any other small narrow-scoped formatters before the Ruff run.
Description of your changes
Subset of #741, to merge before #742 to prevent future violation to be merged, which will requires additional fixes after.
pyproject.toml
as this project never support Python 2